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A STUDIO OF THE VALIDITY OF 
SELF-raPORTED MEASURES OF ACADEMIC GROWTH 

John T. Pohlmann 
Donald L. Beggs I 

Southern Illinois University at Carb^ndale 

This study was undertakafi to eKmine the ifalatlonship between self- 
reported and pre-poit meaaures of academic growth. Self-reported and prs-post 
measures were obtained in three areas, simple cognitlvej compleK cognitive 
and attltudlnal. The subjects were 162 graduate students enrolled in six 
different graduate courses. Partial correlationi relating ielf-reported 
measurei of growth to post test perfomance on measurei of achievement (simple 
and complex GOgnttlve) and attitudes eontrolllng for pre test performance; 
indicated that self -reported meaiures of academic growth were primarily 
related to growth in attitudes toward the subject laattar of a course « 



2 



A STUDY OF THE VALIDITY OF 
SELF -REPORTED MEASURES OF AGADEHIC GROWTH 

The typical course evaluation questionnaire solicits student ratings 
of such factors as instructor performance and course cQntent* As such, 
they serve as a process evaLuatlon of what transptred in the classroom. 
Recently, Hogan and Hartley (1972) have suggested that student ratings 
□f their developraent or chanp^ be incorporated into evaluation questionnaires. 
This approach to the evaluation of courses and instruction has been seen as a Ttieans 
whereby t)ie focus of the evaluatiori ctuld be changed to reflect the outcomes 
of instruction. 

The purpoie of this study was to relate measures of student self ^reported 
academic groi^th, as measured by a scale similar to Hogan and Hartley 's, to 
indices of student devilopmint derived from pre and post test measures of 
cognittva and affective variables related to the content of Instruction. 
Mwasures of self-reported and observed growth were obtained In three domains 
of academic behavior^ 1, simple cognitive, 2, complex cognitive, and 
3. affective. 

PrevLPus research indicated that self-report methods of assessing cognitive 
variables J such as Intelligence or academic ability, are positively related 
to observed measures of the same variables, Berdie (1971) ^ In a study using 
college students, observed correlations between self-claimed and tested 
knowledge of famous people In politics , the artSj and history, ranging from 
.^7 to .74. Bowen (1968) examined the relationship between self-estimates 
of academic ability and academic performance on a sample of 389 high 
school students. He fQund that self-estimates of the ability to do 
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school worl< correlated ,64 with high school grade point averagts Bowen 
also found self-esttmates of academic ability to.be related to compostte 
scoris on the DifferentiaL Aptitude Test Battery (r-,51). 

A number of self-report personality and interest inventories contain 
scales which are keyed against measures of cognitive ability. For eKampLej 
the Intellectual Efficiency (le) scale from the California personality 
Inventory (Gough, 1957) was found to correlate positively with the Terman 
Concept Mastery Test (r^.SS), which is a measure of general intelligence- 
A correlation of ,50 was also observed between le and the Kulman-Anderson 
Intelligence TdSt, 

Self-report meanures have also bean found to be, useful in aisessing 
affective variables, Hamilton (1971) 'found that simple one item self- 
ratings of self-esteem, dominance and open=mlndedness perform as well as 
other methods of measurement^ such as peer nominations and empertcally 
derived scales, in tarms of thetr convergent and discriminant validity. 
In a similar study, Wetgel (1963) related simple one Item self-ratings 
of adjustment and extroversion to two second order factor scores derived 
from CattelPs 16PF which were labeled "dynamic integration" and ■ 
"introversion-extroversion," Self-ratings of adjustment correlated ,56 
with the dynamic integration factor score, and ratings of extroversion 
correlated ,66 with the Introversion-eKtroversion factor score. 
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tffiTHOD = 

S ub 1 a eta 

Tha subjects for this study were 162 graduate students enrolled 
in six different sections of three different statistics courses taught in the 
College of Education at Southern Illinois University at Carbondale, The 
three courses were, 1, an Introductory statistics course, 2* an eKperl- 
mental design coursej and 3, a research design course emphasizing the use 
of the general linear regression model. 
Measures 

Cognitive Domain , Objective tests were developed based on the content 
of each course, a separate test being developed for each course. Items 
included in the tests were taken from a nimber of iources such as the 
instructor's manual for the tert, and items from exmni developed and used 
previously by the instructor smple. Some Items were developed specifically 
for the purposes of this study. Each Instructor reviewed the items and only 
those items which were considered to be measuring cQntent that was actually 
presented were retained* Two kinds of items were developed for each exam. 
One type consisted of items Intended to measure simple cognitive outcomes, such 
as symbol recognition and definition of terms. The format used for these simple 
cognitive items was a series of matching exercises* The second type of Items 
were intended to assess more complex skills such as analysis and problem 
solving. In this section of each exOTij multiple choice item forrnats were 
used. Total scores on each of these sections were considered respectively to 
be measures of simple and coB^lex cognitive achievement* Another measure 
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of student achievement was obtained by using the scores each^ studeiu rt?ceived 
in classroom eKams , When more than one exam was givenj the individual exams 
were standardized and sunmied to obtain a composite index of^ itudenr achievement 
on class exami * 

AffecElve Domalri . A measure of the affective orientatiSh toward 
the content of each courie was obtatnid by using an eleven item semantic 
differential icale on which the students rated the subject matter of the 
course. The items were composed of bipolar adjecttvss that loaded high on 
the evaluation factor deacribid by Osgood Bt al. (1957). 

Self Report ed Growth , Measures of self-reported growth were obtained 
by having the subjects rite, on a five pcint scales the degree of growth 
they felt they had realized as a result of taking the course. The subjects 
were asked to rate their growth by reapondlng to eleven items that 
described various educational outcomes in the cognitive and affective domains. 
The items on the self-reported growth scale deicribed educational outcomes 
which paralleled the Taxonomy of Educational Objectives (Bloom, et al.^ 1971), 
Four Items related to growth in performance on objectives ranging from 
knowledge to application were combined into a scale to measure self- 
reported simple cognitive growth. Three items relating to objectives ranging 
from analysis to evaluation were combined into a scale to measure self =repor ted 
complex cognitive growth. Four items relating attitudes toward end valuing 
of the Subject matter were combined into a scale to measure self ^reported 
affective growth, — 
Procedures. Early in the quarter pre tests were administered with the 
achleviment tesug and the semantic differential attitude scale. During- the 
last week of the quarter, post-tests were administered with these same 
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scales. The sel f -reported growth scales were also adniintstertHl during iho 
poBt testLng session. Approximately ten weeks separated the pre and poHt 
testing sessions. 

M issing Data . Out of the original sample of 162 subjects, 133 
generated sufficient data for the final analysts. Fifteen subjects 
withdrew from the courses, eight failed to complete the test battery, or 
filled out the answer sheets improperly, and six persons did not atterid 
class at the time of the post test. 

Statistical Treatment of Giowth Data , In keeping with the position 
taken byO^Connor (1972)5 the relationship between self-reported growth 
and pre-post growth was expressed in terms of a partial correlation 
coefficient. Each self-report measure was correlated with the appropriate 
post test memsure of performance, with the correspondlhg pre test measure 
being statistically held constants Ihe obtained partial correlations 
were then tested to determine if they were significantly greater th&n zero (C^ *.05), 
A one tailed alternative was used since the review of literature suggested 
that positive relationships would be observed. 

RESULTS 

Table 1 contains the partial correlations between each of the self- 
report measures and post test performance , controlling for pre test 
performance* the first row In Table 1 presents the partial r*s for 
simple cognitive growth, and they represent the correlation between self- 
reported simple cognitive growth and post test simple cognitive achievement, 
With the exception of class 2 the correUtiona are small and positive. 



The pooled within clais partial r was significantly greater than zero, 
but when class 2 was removed from the analyses, the pooled within partial 
r reduced to ,14 (not significant). 

The second row in Table 1 contains the partial r's for complex cognitive 
growth. They represent the correlation between self-reported compliK 
cognitive growth and post test performance on the complex cognitive 
achievement test, controlling for pre test performance on the same exam, 
None of the within class correlations were significant^ nor was the pooled 
within class partial r significantly greatsr than zero* 

The coefficients in the third row of Table 1 are the partial r^s 
relating self-reported growth in attitudes to post test attltude^core, 
controlling for pre test attitude score. The partial r-s were significantly 
greater than zero in each class* 

The last row of cnefflcients in Table 1 are the partial r's relating 
self -reported cognitive growth (simple and compleK combined) and performance 
on class exams , Pre test composite performance on the simple and compleK 
cognitive achievement tests was partlaled out. When more than one class 
exam was given, the scores on each eKam were standardized and summed to 
obtain a composite indeK of achievement on class exams. The partial r for 
class 4 and the pooled within class partial r were significantly greater 
than zero. J ' 
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DISCUSSION 

These results suggest that self^report meaauras of academic growth 
are primarily sensitive to growth in s tudent » s affective orientation 
toward the subject matter of the course.. While positive correlations wer 
observed between self-reported and pre-post measures of cognitive growth, 
they tended to be low. Consequently, evaluators using student ratings 
of cognitive growth should be encouraged to also collect pre-post 
performance data relative to the objectives of the course, since these 
two measures of growth (self-reported and pre-post) are relatively 
independsnt. 
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